Programming device

ABSTRACT

Provided is a programming device capable of reducing the load of an operator that creates a control program including a command corresponding to a function of an imaging device. Provided is a programming device for programming industrial machinery, the programming device including: a command generation unit that acquires information regarding an imaging device connected to a control device of the industrial machinery and that, on the basis of the acquired information, generates an icon or command statement expressing a command using an image acquired by the imaging device; and a command display unit that displays the generated icon or command statement on a display screen.

CROSS REFERENCE TO RELATED APPLICATIONS

This is the U.S. National Phase application of PCT/JP2021/020897, filedJun. 1, 2021 which claims priority to Japanese Patent Application No.2020-098761, filed Jun. 5, 2020, the disclosures of these applicationsbeing incorporated herein by reference in their entireties for allpurposes.

FIELD OF THE INVENTION

The present invention relates to a programming device for an industrialmachine.

BACKGROUND OF THE INVENTION

Since it is a common practice to perform programming of a robot using atext-based statement, an operator may preferably have knowledge of aprogramming language for the robot. To support intuitive input by anoperator of a control program for a robot, a program creation devicethat allows programming using an icon representing each statement ofrobot control has been proposed (see, e.g., PTL 1).

PATENT LITERATURE

-   [PTL 1] Japanese Patent No. 6498366 B

SUMMARY OF THE INVENTION

A set of instructions for programming in a robot system including animaging device includes an instruction corresponding to a function forcapturing an image of a workpiece using the imaging device, processingthe captured image, and detecting the workpiece. Generally, when astatement corresponding to such a function that uses an imaging deviceis included in a control program for a robot, an operation for insertinga statement corresponding to a function that uses an imaging device intoa control program for a robot may be preferably performed, and then animaging device to be started in accordance with the statement may bepreferably selected and set separately. The number of imaging devicesplaced in the robot system may be one or plural. In the latter case, anoperator may preferably insert a plurality of statements correspondingto the functions of the imaging devices into the program, and thenrepeat selection of an imaging device to be started in accordance witheach inserted statement, thus inflicting a heavier burden on theoperator. In programming of a control program for an industrial machine,a demand has arisen for a programming device that can reduce the burdenon an operator who creates a control program including an instructioncorresponding to the function of an imaging device.

One aspect of the present disclosure provides a programming device forperforming programming of an industrial machine, the device including aninstruction generation section configured to acquire informationconcerning an imaging device connected to a controller for theindustrial machine, and generate one of an icon and a statementrepresenting an instruction to use an image captured by the imagingdevice, based on the acquired information, and an instruction displaysection configured to display the generated one of the icon and thestatement on a display screen.

With the above-mentioned configuration, in programming of a controlprogram for an industrial machine, it is possible to reduce the burdenon an operator who creates a control program including an instructioncorresponding to the function of an imaging device.

These and other objects, features, and advantages of the presentinvention will become more apparent from the following detaileddescription of an exemplary embodiment thereof, as illustrated in theaccompanying drawings, in which:

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram depicting the configuration of a robot systemincluding a teaching device (programming device) according to oneembodiment.

FIG. 2 is a functional block diagram of the teaching device.

FIG. 3 is a flowchart illustrating instruction generation and displayprocessing.

FIG. 4 illustrates a first example of generation and display of an iconof an imaging device by the instruction generation and displayprocessing.

FIG. 5 illustrates a second example of generation and display of an iconof an imaging device by the instruction generation and displayprocessing.

FIG. 6 illustrates an exemplary screen displayed when a Details tab ispressed in a program creation screen.

FIG. 7 illustrates an exemplary program creation screen for detailedprogramming associated with the function of an imaging device.

FIG. 8 is a diagram illustrating an exemplary text-based programcreation screen in a general teaching device, as a comparative example.

FIG. 9 is a diagram illustrating an exemplary text-based programcreation screen in this embodiment.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

An embodiment of the present disclosure will be described below withreference to the drawings. In the drawings to be referred to, the sameor similar reference numerals denote the same or similar components orfunctional parts. To facilitate understanding, these drawings usedifferent scales as appropriate. Further, the modes illustrated in thedrawings are merely examples for carrying out the present invention,which is not limited to the modes illustrated in the drawings.

FIG. 1 is a diagram depicting the configuration of a robot system 100including a teaching device (programming device) 10 according to oneembodiment. The robot system 100 includes a robot 30 having a hand 31mounted at its arm distal end, vision sensors 41 and 42, a robotcontroller 20 that controls the robot 30, and the teaching device 10connected to the robot controller 20, as illustrated in FIG. 1 . Therobot 30 is implemented as a vertical articulated robot in this case,but other types of robots may be used. The robot controller 20 controlsthe motion of the robot 30 in accordance with a control program, or inresponse to a command issued from the teaching device 10. The robotcontroller 20 may have the configuration of a general computerincluding, e.g., a CPU, a ROM, a RAM, a storage device, an operationsection, a display section, an I/O interface, and a network interface.

The vision sensor 41 is mounted at the arm distal end of the robot 30.The vision sensor 42 is fixed at a position that allows capture of animage of a workpiece 1 in a workspace in which the robot 30 isinstalled. In the robot system 100, the position of the workpiece 1 isdetected by the vision sensors 41 and 42, and the workpiece 1 placed ona work table 2 is handled by the hand 31 of the robot 30.

The vision sensors 41 and 42 are connected to the robot controller 20and operate under the control of the robot controller 20. The robotcontroller 20 also includes the function of an image processing devicethat acquires images captured by vision sensors (vision sensors 41 and42) and perform image processing such as workpiece detection for thecaptured images. The robot controller 20 holds a model pattern for aworkpiece, and detects the workpiece by pattern matching between theworkpiece in the captured images and the model pattern. The robotcontroller 20 can compensate a teaching position based on the detectedposition of the workpiece 1 and perform, e.g., picking of the workpiece1.

The vision sensors 41 and 42 may serve as cameras (2D cameras) thatcapture grayscale images or color images, or may serve as stereo camerasor three-dimensional sensors (3D cameras) that can capture range imagesor three-dimensional point clouds. As an example, this embodimentassumes that the vision sensor 41 serves as a 2D camera, and the visionsensor 42 serves as a 3D camera. FIG. 1 illustrates an example in whicha total of two imaging devices: one 2D camera and one 3D camera areplaced in the robot system 100, but the number of imaging devicesprovided in the robot system 100 may be one, or three or more. Inaddition, a plurality of imaging devices of the same type (e.g., aplurality of 2D cameras or a plurality of 3D cameras) may be placed inthe robot system 100.

The teaching device 10 is used to create a control program for causingthe robot 30 to handle the workpiece 1. The teaching device 10 may beimplemented as, e.g., a tablet terminal or a teach pendant connected tothe robot controller 20. Alternatively, the teaching device 10 may beimplemented as a programming device (e.g., a PC) for performingprogramming offline. The teaching device 10 may have the hardwareconfiguration of a general computer including, e.g., a CPU, a ROM, aRAM, a storage device, an I/O interface, and a network interface.

FIG. 2 is a functional block diagram of the teaching device 10. FIG. 2also depicts the connection relationship between the teaching device 10and other devices. The teaching device 10 includes an instructiongeneration section 11, an instruction display section 12, a programediting section 13, and a program generation section 14, as illustratedin FIG. 2 .

The instruction generation section 11 acquires information concerningone or more imaging devices connected to the robot controller 20 fromthe robot controller 20, and generates an icon or a statementrepresenting an instruction to use an image captured by each of the oneor more imaging devices, in a representational form that allowsidentification of each of the one or more imaging devices, based on theacquired information. The “information concerning an imaging device”means herein information (e.g., a product model) that allowsidentification of the type of each imaging device connected to the robotcontroller 20. Hence, the instruction generation section 11 canidentify, e.g., a 2D camera (monochrome), a 2D camera (color), or a 3Dcamera as the type of imaging device provided in the robot system 100.

The “information concerning an imaging device” may further includeinformation that allows identification of the installation position ofthe imaging device. An example of the information that allowsidentification of the installation position may be informationindicating whether the imaging device serves as a camera (handheldcamera) mounted on a robot, or a fixed camera fixed in the workspace.Since a connection cable for a camera mounted on a robot is generallywired through the interior of the robot and connected to a specific I/Ointerface of a robot controller, the robot controller can identify theinstallation position of the imaging device (whether the imaging deviceserves as a handheld camera or a fixed camera), based on which I/Ointerface the imaging device is connected to (e.g., informationindicating which port the imaging device is connected to). More detailedposition information (e.g., coordinate information of the imaging devicein the workspace when the imaging device serves as a fixed camera) maybe provided as the information representing the installation position ofthe imaging device.

Alternatively, when a robot is shipped upon mounting of an imagingdevice on the robot, information indicating that the camera used servesas, e.g., a handheld camera may be held in this camera as part ofproduct information. In the robot system 100 having the configurationillustrated in FIG. 1 , the instruction generation section 11 acquires,e.g., the following information from the robot controller 20 asinformation concerning the vision sensors 41 and 42:

Information Concerning Vision Sensor 41: 2D Camera (Color); HandheldCamera

Information Concerning Vision Sensor 42: 3D Camera; Fixed Camera

Upon acquiring the “information concerning an imaging device” asdescribed above, the instruction generation section 11 can generate anicon or a statement of each imaging device in a representational formthat allows identification of the types, the number, and theinstallation positions of imaging devices connected to the robotcontroller 20. The instruction display section 12 displays theabove-mentioned icon or statement generated by the instructiongeneration section 11 in a predetermined display area of a displaydevice 18.

The program editing section 13 displays, on the display device 18, anediting screen (program creation screen) for creating a control programfor the robot 30, and receives an editing operation for the controlprogram. The control program includes herein an instruction forcontrolling the robot 30, and an instruction associated with capture ofan image by a vision sensor and processing of the captured image. Theprogram generation section 14 generates a control program from an iconplaced in a program creation area within the editing screen, or astatement described in the program creation area within the editingscreen.

Two examples related to generation and display of an icon or a statementby the teaching device 10 will be given below. Example 1 illustrates anoperation example when the teaching device 10 generates and displays anicon representing an instruction to use an image captured by an imagingdevice, and Example 2 illustrates an operation example when the teachingdevice 10 generates and displays a statement (text) representing aninstruction to use an image captured by an imaging device. FIG. 3 is aflowchart illustrating processing (to be also referred to as“instruction generation and display processing” hereinafter) forgenerating and displaying an icon or a statement representing aninstruction to use an image captured by an imaging device. Since theflowchart illustrated in FIG. 3 is common to Examples 1 and 2, these twoexamples will be described below with reference to the flowchartillustrated in FIG. 3 , as appropriate. The processing illustrated inFIG. 3 is performed under the control of the CPU of the teaching device10.

Example 1

In Example 1, the teaching device 10 generates and displays an iconrepresenting an instruction to use an image captured by an imagingdevice. In this case, the instruction generation section 11 functions asan icon generation section, and the instruction display section 12functions as an icon display section. Upon the start of the teachingdevice 10, or in response to a predetermined operation performed on theteaching device 10 to open a program creation screen, the instructiongeneration and display processing illustrated in FIG. 3 is started. Theteaching device 10 (program editing section 13) first displays a programcreation screen (editing screen) 400 on the display device 18 (step S1).FIGS. 4 and 5 illustrate two examples of the program creation screen 400created in this case. The program creation screen 400 includes a programcreation area 300 and an icon display area 200, as illustrated in FIGS.4 and 5 .

The teaching device 10 (instruction generation section 11) then acquiresinformation concerning an imaging device connected to the robotcontroller 20 from the robot controller 20 (step S2). The teachingdevice 10 (instruction generation section 11) generates an icon of theimaging device, based on the information concerning the imaging device(step S3).

The teaching device 10 (instruction generation section 11) confirmswhether icons have been generated for all imaging devices connected tothe robot controller 20, based on the information concerning the imagingdevice (step S4). The teaching device 10 (instruction generation section11) repeats step S3 (NO in step S4), in which an icon is generated,until icons are generated for all imaging devices connected to the robotcontroller 20. When icons have been generated for all imaging devicesconnected to the robot controller 20 (YES in step S4), the processadvances to step S5.

In step S5, the teaching device 10 (instruction display section 12)displays the icons generated by the above-mentioned processing in apredetermined display area (icon display area 200) (step S5). The iconsreferred to in this specification may include all graphicalrepresentations of instructions to various devices in the form of, e.g.,figures, patterns, or symbols. Therefore, the icons include not only theexamples given in this embodiment, but also, e.g., graphic objects(e.g., blocks used in graphic programming in puzzle form) like thoseused in a visual programming tool.

Two examples of generation and display of an icon of an imaging deviceby the instruction generation and display processing illustrated in FIG.3 will be given below with reference to FIGS. 4 and 5 .

FIG. 4 illustrates a first example of generation and display of an iconof an imaging device by the instruction generation and displayprocessing. The program creation screen 400 includes an icon displayarea 200 for displaying a list of icons that can be used forprogramming, and a program creation area 300 for creating a controlprogram by arranging the icons in the order of motion, as illustrated inFIG. 4 . Assume herein that the vision sensor 41 serves as a 2D camera,and the vision sensor 42 serves as a 3D camera. In step S3, the teachingdevice 10 (instruction generation section 11) generates an icon 201having a figure representing the appearance of a 2D camera, as an iconof the vision sensor 41 (2D camera). The icon 201 may be added with atext “Pick Up by Viewing with 2D Camera A” as words expressing thefunction of this icon, as illustrated in FIG. 4 . The teaching device 10(instruction generation section 11) further generates an icon 202 havinga figure representing the appearance of a 3D camera, as an icon of thevision sensor 42 (3D camera). The icon 202 may be added with a text“Pick Up by Viewing with 3D Camera A” as words expressing the functionof this icon, as illustrated in FIG. 4 .

The teaching device 10 (instruction display section 12) displays theicons 201 and 202 in the icon display area 200, together with icons 101to 104 for control (control program) of the robot 30. Since the icons201 and 202 are designed to allow identification of the vision sensor 41(2D camera) and the vision sensor 42 (3D camera), the operator caninstantaneously know the types and number of available imaging devices(i.e., imaging devices connected to the robot controller 20) by viewingthe icons 201 and 202 displayed in the icon display area 200. Theprogram creation screen 400 may be provided with a button for performingan operation for switching between display and non-display of textinformation (e.g., “Catch” or “Release”) added to each of the icons 101to 104, 201, and 202.

The icons 101 to 104, 201, and 202 displayed in the icon display area200 represent the following operation instructions:

Icon 101: Instruction to Close Hand and Catch Workpiece Icon 102:Instruction to Open Hand Icon 103: Instruction to Move Arm Distal End ofRobot in Linear Path Icon 104: Instruction to Move Arm Distal End ofRobot in Circular Path Icon 201: Detect Position of Workpiece by 2DCamera Icon 202: Detect Position of Workpiece by 3D Camera

The program editing section 13 receives an operation, by the operator,for placing the icons displayed in the icon display area 200 in theprogram creation area 300 by a drag-and-drop operation. The programgeneration section 14 generates a control program in accordance with theicons placed in the program creation area 300. In the program creationarea 300 illustrated in FIG. 4 , the icon 103, the icon 201, the icon101, the icon 103, and the icon 104 are arranged in the order of motion.In this control program, a motion is executed in which the arm distalend of the robot 30 is linearly moved to a target position, the positionof a workpiece is detected by a 2D camera, a teaching position iscompensated based on the detected position, and the workpiece is grippedand moved by the hand 31.

FIG. 5 illustrates a second example of generation and display of an iconof an imaging device by the instruction generation and displayprocessing. Assume herein that both of the vision sensors 41 and 42serve as 2D cameras. In this case, information concerning an imagingdevice acquired from the robot controller 20 by the teaching device 10(instruction generation section 11) includes not only the type ofimaging device, but also information concerning the installationposition of the imaging device. The teaching device 10 (instructiongeneration section 11) determines that the vision sensor 41 serves as ahandheld camera placed at the arm distal end of the robot 30, and thevision sensor 42 serves as a fixed camera fixed in the workspace, basedon the information concerning the imaging device acquired from the robotcontroller 20.

With this operation, the teaching device 10 (instruction generationsection 11) generates an icon 212 including a figure that reminds theoperator that the vision sensor 41 serves as a camera mounted at the armdistal end of a robot, as an icon of the vision sensor 41. The icon 212may be added with words (in this case, “Pick Up by Viewing with 2DHandheld Camera”) that facilitate understanding of the function and theinstallation position indicated by the icon 212, as illustrated in FIG.5 . The teaching device 10 (instruction generation section 11) furthergenerates an icon 211 having a figure that reminds the operator that thevision sensor 42 serves as a fixed camera, as an icon of the functionimplemented by the vision sensor 42. The icon 211 may be added withwords (in this case, “Pick Up by Viewing with 2D Fixed Camera”) thatfacilitate understanding of the function and the installation positionindicated by the icon 211, as illustrated in FIG. 5 .

The teaching device 10 (instruction display section 12) displays theicons 211 and 212 in the icon display area 200, together with icons 101to 104 for control of the robot 30. Since the icons 211 and 212 aredesigned to allow identification of the vision sensor 41 (handheldcamera) and the vision sensor 42 (fixed camera), the operator caninstantaneously know the types, number, and installation positions ofavailable imaging devices (i.e., imaging devices connected to the robotcontroller 20) by viewing the icons 211 and 212 displayed in the icondisplay area 200.

The program editing section 13 receives an operation, by the operator,for placing the icons displayed in the icon display area 200 in theprogram creation area 300 by a drag-and-drop operation. The programgeneration section 14 generates a control program in accordance with theicons placed in the program creation area 300. In the program creationarea 300 illustrated in FIG. 5 , the icon 103, the icon 211, the icon101, the icon 103, and the icon 104 are arranged in the order of motion.In this control program, a motion is executed in which the arm distalend of the robot 30 is linearly moved to a target position, the positionof a workpiece is detected by a 2D fixed camera, a teaching position iscompensated based on the detected position, and the workpiece is grippedand moved by the hand 31.

An operation executed when detailed programming is performed for theicons (icons 201 and 202 or icons 211 and 212) associated with thefunction of an imaging device (an instruction to use an image capturedby an imaging device) generated by the instruction generation section 11will be described below. FIG. 6 illustrates, as the program creationscreen 400 as illustrated in FIG. 4 or 5 , an exemplary program creationscreen 400 in which the operator selects an icon (e.g., the icon 201)associated with the function of an imaging device placed in the programcreation area 300, and presses a details tab 262 provided in the programcreation screen 400. In this case, a create button 501 and a drop-downmenu 502 appear in the lower region of the program creation screen 400,as illustrated in FIG. 6 . In creating a new detailed program associatedwith the function of an imaging device, the operator presses the createbutton 501. A program associated with the function of an imaging deviceregistered in the teaching device 10 is displayed in the drop-down menu502. The operator can open a desired program by selecting this programfrom the drop-down menu 502 and pressing an open button 503.

Alive image 550 of an imaging device corresponding to the icon (e.g.,the icon 201) selected in the program creation area 300 may be displayedin the lower region of the program creation screen 400 illustrated inFIG. 6 . With such a configuration, the operator can more reliablyrecognize which imaging device corresponds to the icon selected in theprogram creation area 300 by viewing the live image 550. Such a displayfunction of the live image 550 may be implemented by, e.g., using theprogram editing section 13 to request of the robot controller 20transmission of a live image of an imaging device corresponding to theicon selected in the program creation area 300.

When a programming tab 261 is selected in the program creation screen400 illustrated in FIG. 6 , the lower region of the program creationscreen 400 returns to the icon display area 200 that displays a list oficons, and a return, in turn, is made to the state in which the operatorcan perform programming of a robot.

FIG. 7 illustrates a program creation screen 400A for detailedprogramming associated with the function of an imaging device, displayedupon pressing of the create button 501 or the open button 503 in thescreen illustrated in FIG. 6 . The program creation screen 400Aassociated with the function of an imaging device also has the samescreen configuration as that of the program creation screen 400 used increating a control program for a robot, as illustrated in FIG. 7 . Theprogram creation screen 400A includes an icon display area 200A fordisplaying an icon associated with control of an imaging device, and aprogram creation area 300A for creating a detailed program for afunction associated with the imaging device. As in the case where acontrol program for a robot is edited, the operator can create adetailed program associated with the function of an imaging device byselecting icons from the icon display area 200A, and arranging them inthe program creation area 300A by a drag-and-drop operation.

In the example illustrated in FIG. 7 , the icons displayed in the icondisplay area 200A include an imaging icon 252 representing a functionassociated with imaging, a detection icon 253 representing a functionfor detecting a workpiece, and a compensation calculation icon 254representing calculation of an amount of compensation. The operator canperform a predetermined operation by selecting each icon placed in theprogram creation area 300A, display a parameter setting screen in thelower portion of the program creation screen 400A, and perform detailedparameter setting. Parameter setting of the imaging icon 252 isperformed for the following setting items:

Exposure Time

ON or OFF of LED Illumination

Image Reduction Ratio

Parameter setting of the detection icon 253 is performed for thefollowing setting items:

Image Used

Shape to Be Found

Threshold for Degree of Matching

Threshold for Contrast

where “Threshold for Degree of Matching” and “Threshold for Contrast”are parameters associated with thresholds in image processing fordetection of a workpiece.

In the compensation calculation icon 254, an amount of compensation forcompensating a teaching position of a robot is obtained by, e.g.,calculating the position of a workpiece in the image, based on thedetection result obtained by the detection icon 253, and converting thecalculated position in the image into 3-dimensional coordinates in arobot coordinate system.

In a program (to be referred to as a vision program 604 hereinafter) forthe program creation area 300A illustrated in FIG. 7 , an imaging icon252 is placed first, an auxiliary icon 257 corresponding to a detectionoperation is then placed, and two detection icons 253 are placed in theframe of the auxiliary icon 257. The vision program 604 implements thefollowing operations: First, an image of a workpiece is captured by theimaging icon 252. Then, the shape of the entire workpiece is detected bythe auxiliary icon 257 for detection, and a feature (e.g., a hole) onthe workpiece is detected by the two detection icons 253 in theauxiliary icon 257. This makes it possible to detect a workpiece havingtwo holes formed in it. The coordinates of the workpiece in the robotcoordinate system are calculated and set as data for positioncompensation by the compensation calculation icon 254.

Example 2

Example 2 related to generation and display of an icon or a statement bythe teaching device 10 will be given below. In Example 2, the teachingdevice 10 generates and displays a statement (text) representing aninstruction to use an image captured by an imaging device. FIG. 9illustrates an exemplary text-based program creation screen 400Bgenerated by the teaching device 10 according to this embodiment, andFIG. 8 illustrates an exemplary text-based program creation screen 140in a general teaching device, as a comparative example. FIGS. 8 and 9both illustrate program creation screens for executing the same motionin a robot system including two imaging devices as illustrated in FIG. 1.

FIG. 9 illustrates the program creation screen 400B displayed by theprogram editing section 13 of the teaching device 10 according to thisembodiment. The program creation screen 400B includes a program creationarea 300B for performing text-based programming. A popup menu 200B fordisplaying a list of statements can be displayed by performing apredetermined operation (e.g., an operation for pressing an instructionlist button (not illustrated) displayed in the program creation screen400B) via an input device 19 of the teaching device 10, with the programcreation screen 400B being displayed. By moving a cursor to a positionhaving a desired line number in the program creation area 300B, andselecting a desired statement from the popup menu 200B, the operator caninsert the statement at the position of the cursor.

Generation and display processing of a statement by the teaching device10 will be described below with reference to the flowchart illustratedin FIG. 3 . In the beginning, upon the start of the teaching device 10,or in response to a predetermined operation performed on the teachingdevice 10 to open a program creation screen, the processing illustratedin FIG. 3 is started. The teaching device 10 (program editing section13) first opens the program creation screen 400B (step S1). Theinstruction generation section 11 then acquires information concerningan imaging device connected to the robot controller 20 from the robotcontroller 20 (step S2). In this case, information of two vision sensorsis acquired. The two cameras are defined as cameras A and B, for thesake of convenience. The instruction generation section 11 generates astatement for each of the two cameras A and B. The instructiongeneration section 11, for example, generates statements “CAMERA_AVISION RUN_FIND” and “CAMERA_B VISION RUN_FIND” for issuing commands tothe cameras A and B, respectively, regarding a statement “VISIONRUN_FIND” serving as an instruction to capture an image by an imagingdevice (step S3). The instruction generation section 11 furthergenerates statements “CAMERA_A VISION GET_OFFSET” and “CAMERA_B VISIONGET_OFFSET” for issuing commands to the cameras A and B, respectively,regarding a statement “VISION GET_OFFSET” serving as an instruction todetect the position of a workpiece from the captured image (steps S3 andS4).

The instruction display section 12, for example, includes, in the popupmenu 200B, not only a statement associated with control of a robot, butalso the statements “CAMERA_A VISION RUN_FIND,” “CAMERA_B VISIONRUN_FIND,” “CAMERA_A VISION GET_OFFSET,” and “CAMERA_B VISIONGET_OFFSET” associated with the functions of the imaging devicesgenerated by the instruction generation section 11, and displays thesestatements, in response to a selection operation on an instruction listbutton (not illustrated) (step S5). The operator can instantaneouslyknow the types and number of imaging devices connected to the robotsystem by viewing such a popup menu 200B (a list of statements). FIG. 9illustrates the case where the two cameras are represented as “CAMERA_A”and “CAMERA_B,” but, for example, the portion of “CAMERA_A” may berepresented as “2D_HANDHELD_CAMERA,” and the portion of “CAMERA_B” maybe represented as “3D_FIXED_CAMERA,” so that “2D_HANDHELD_CAMERA VISIONRUN_FIND,” “3D_FIXED_CAMERA VISION RUN_FIND,” “2D_HANDHELD_CAMERA VISIONGET_OFFSET,” and “3D_FIXED_CAMERA VISION GET_OFFSET” are included in thepopup menu 200B as statements. In this case, the operator can know thetypes, the number, and the installation positions of imaging devicesmounted in the robot system 100.

The operator creates a control program as described in the programcreation area 300B by repeating an editing operation for selecting adesired statement from the list of statements displayed in the popupmenu 200B and inserting it into a line pointed to by the cursor. In thiscase, the position of a workpiece is detected by the camera A inaccordance with statements (lines 3 and 4) defined to directly specifythe camera A, and a motion for gripping the workpiece while compensatingthe position of a robot, based on the detected position, is implemented(lines 6 to 10). In addition, the position of the workpiece is detectedby the camera B in accordance with statements (lines 13 and 14) definedto directly specify the camera B, and a motion for gripping theworkpiece while compensating the position of the robot, based on thedetected position, is implemented (lines 16 to 20).

A general teaching device performs no processing for generating astatement for each imaging device. Therefore, as illustrated in FIG. 8 ,in the program creation screen 140 used by the general teaching device,a popup menu 120 for displaying a list of statements displays onlystatements “VISION RUN_FIND” and “VISION GET_OFFSET” defined to specifyno imaging device, as statements associated with an imaging device. Inthis case, the operator, for example, may preferably select thestatement “VISION RUN_FIND” from the popup menu 120, insert it into aprogram creation area 130, and then select a camera used in thestatement “VISION RUN_FIND” as a parameter. In such a popup menu 120,the operator may not be allowed to know the types, the number, and theinstallation positions of imaging devices mounted in the robot system.

The case where a plurality of imaging devices are placed in the robotsystem has been described in the above-described embodiment, but evenwhen only one imaging device is placed in the robot system, an icon withthe imaging device being selected is displayed in the icon display area200 in Example 1 (FIGS. 4 and 5 ), and a statement with the imagingdevice being selected is displayed in the popup menu 200B in Example 2(FIG. 9 ). For example, in Example 1, when only one imaging device(vision sensor 41) is placed in the robot system, the icon 201 isdisplayed in the icon display area 200. In Example 2, when only oneimaging device (camera A) is placed in the robot system, “CAMERA_AVISION RUN_FIND” and “CAMERA_A VISION GET_OFFSET” are displayed in thepopup menu 200B. Therefore, even when only one imaging device is placedin the robot system, the operator may preferably perform no operationfor selecting a camera as detailed setting for an icon or a statement.

Hence, according to this embodiment, in programming of a control programfor an industrial machine, it is possible to reduce the burden on anoperator who creates a control program including an instructioncorresponding to the function of an imaging device.

Although the present invention has been described above with referenceto exemplary embodiments, it will be appreciated by those skilled in theart that the foregoing and various other changes, omissions, oradditions may be made to the above-described embodiments withoutdeparting from the scope of the invention.

The above-described embodiment is related to creation of a controlprogram in a robot system, but it is applicable to a programming devicefor a control program in a system formed by connecting one or moreimaging devices to a controller for any of a machine tool and variousother industrial machines.

The functional blocks of the teaching device 10 illustrated in FIG. 2may be implemented by executing various types of software stored in thestorage device by the CPU of the teaching device 10, or may beimplemented by a configuration mainly formed by hardware such as an ASIC(Application Specific Integrated Circuit).

The functional layout of the functional block diagram illustrated inFIG. 2 is merely an example, and some of the functions arranged in theteaching device 10 may be arranged in the robot controller 20.

The timing at which the teaching device 10 (instruction generationsection 11) acquires information concerning an imaging device from therobot controller 20 and generates an icon or a statement is not limitedto the example given in the above-described embodiment, and variousexamples are available such as the timing at which an imaging device isconnected to the robot controller 20. The timing at which the teachingdevice 10 (instruction generation section 11) acquires informationconcerning an imaging device from the robot controller 20 and generatesan icon or a statement may be set to one of predetermined start timingssuch as the time of the start of the teaching device 10, the time ofconnection of an imaging device to the robot controller 20, and the timeat which the program creation screen 400 is opened, or may be set to aplurality of timings among these start timings.

A program for executing the instruction generation and displayprocessing of FIG. 3 in the above-described embodiment can be recordedon various computer-readable recording media (e.g., a ROM, an EEPROM, asemiconductor memory such as a flash memory, a magnetic recordingmedium, or an optical disk such as a CD-ROM or a DVD-ROM).

REFERENCE SIGNS LIST

-   1 Workpiece-   2 Work table-   10 Teaching device-   11 Instruction generation section-   12 Instruction display section-   13 Program editing section-   14 Program generation section-   18 Display device-   19 Input device-   20 Robot controller-   30 Robot-   41, 42 Vision sensor-   31 Hand-   200 Icon display area-   200B Popup menu-   300, 300B Program creation area-   400, 400B Program creation screen

1. A programming device for performing programming of an industrialmachine, the device comprising: an instruction generation sectionconfigured to acquire information concerning an imaging device connectedto a controller for the industrial machine, and generate one of an iconand a statement representing an instruction to use an image captured bythe imaging device, based on the acquired information; and aninstruction display section configured to display the generated one ofthe icon and the statement on a display screen.
 2. The programmingdevice according to claim 1, wherein the instruction generation sectionacquires the information concerning the imaging device from thecontroller connected to the programming device, at least at one ofpredetermined start timings including a time of start of the programmingdevice, a time of connection of the imaging device to the controller,and a time at which a program creation screen is opened.
 3. Theprogramming device according to claim 1, wherein the instructiongeneration section acquires the information concerning the imagingdevice from the controller connected to the programming device, inresponse to a predetermined operation performed to start a programcreation screen via an input device of the programming device.
 4. Theprogramming device according to claim 1, wherein the informationconcerning the imaging device includes information concerning at leastone of a type of the imaging device, an installation position of theimaging device, and a port connected to the imaging device.
 5. Theprogramming device according to claim 4, wherein the informationconcerning the imaging device comprises information concerning aplurality of imaging devices connected to the controller, and theinstruction generation section generates one of the icon and thestatement representing an instruction to use an image captured by eachof the plurality of imaging devices, in a representational form thatallows identification of the each of the plurality of imaging devices.6. The programming device according to claim 5, wherein the informationconcerning the imaging device includes information concerning a type ofeach of the plurality of imaging devices, and the instruction generationsection generates one of the icon and the statement representing aninstruction to use an image captured by each of the plurality of imagingdevices, in a representational form that allows identification of thetype of the each of the plurality of imaging devices.
 7. The programmingdevice according to claim 5, wherein the information concerning theimaging device includes information concerning an installation positionof each of the plurality of imaging devices, and the instructiongeneration section generates one of the icon and the statementrepresenting an instruction to use an image captured by each of theplurality of imaging devices, in a representational form that allowsidentification of the installation position of the each of the pluralityof imaging devices.
 8. The programming device according to claim 1,further comprising a program editing section configured to display aprogram creation screen for performing the programming and receive anediting operation on the program creation screen, wherein theinstruction generation section generates the icon representing theinstruction to use the image captured by the imaging device, theinstruction display section displays the generated icon in apredetermined display area within the program creation screen, theprogram creation screen includes a program creation area for creating aprogram by placing an icon selected from the predetermined display areain the program creation area, and the program editing section displays,in response to an operation for selecting the icon placed in the programcreation area, a live image of the imaging device corresponding to theselected icon.